A Keyword Based Prototype for Web Search Result Diversification

In web search scenario, users often submit short query terms to search engines, expecting to find their desired information in top ranked results. But their queries are so ambiguous that their actual information needs are often unspecified. To satisfy the different information needs, an effective approach is to diversify the top results retrieved for the query. In this paper, we reduce the diversification problem into optimizing the maximum coverage of information facets related to the query, and introduce KED, a novel keyword based prototype for web search result diversification that provides a diverse ranking by selecting documents to cover keywords which belong to different facets underlying the retrieved documents. We evaluated the effectiveness of KED using two public test collections with different kinds of documents. The experiment results show that KED can stably outperform other existing implicit diversification approaches in promoting diversity of top ranked results. Moreover, we show that its effectiveness can be further improved by using high quality keywords.